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ANALYSIS  OF  COMPLEX  THREATS  II 
SUMMARY 


THE  PROJECT  PURPOSE  was  to  deepen  and  extend  the  methodology  developed  in  Analysis 
of  Complex  Threats  (ACT)  (CAA-SR-99-4)  in  order  to  validate  a  15-year  forecast  of  country 
instability. 

THIS  PROJECT  WAS  COSPONSORED  by  the  Office  of  the  Deputy  Chief  of  Staff  for 
Operations  and  Plans  (ODCSOPS),  War  Plans  Division,  and  the  Office  of  the  Deputy  Chief  of 
Staff  for  Intelligence  (ODCSINT),  Headquarters,  Department  of  the  Army. 

THE  SCOPE  OF  THE  PROJECT  was  to: 

(1)  Broaden  the  scope  of  the  country  instability  analysis  first  conducted  in  the  ACT  study  to 
include  additional  dependent  and  independent  variables; 

(2)  Continue  the  examination  of  available  analytical  methods  as  well  as  the  development 
and  refinement  of  new  methods  for  forecasting  country  instability; 

(3)  Conduct  and  validate  a  15-year  forecast  of  country  instability. 

THE  BASIC  APPROACH  for  this  project  was  to 

(1)  Expand  the  temporal  domain  of  the  ACT  database  from  1989-1997  to  1960-1999  in 
order  to  provide  sufficient  historical  data  to  validate  a  long-term  forecast  of  country  instability; 

(2)  Add  additional  macro-structural  indicators  to  the  database  including  levels  of 
democracy,  global  trade,  average  life  expectancy,  and  religious  and  ethnic  diversity; 

(3)  Add  a  dependent  variable  to  the  database  that  tracks  both  violent  and  nonviolent  intra- 
and  interstate  conflicts  that  have  occurred  over  the  period  1945-1999; 

(4)  Refine  the  Fuzzy  Analysis  of  Statistical  Evidence  (FASE)  methodology,  first  used  in 
ACT,  to  predict  various  levels  of  intensity  of  country  instability. 

THE  PRINCIPAL  FINDINGS  are  that: 

(1)  A  new  dependent  variable  was  successfully  incorporated  into  the  approach  to  allow  for 
forecasting  different  levels  of  intensity  of  country  instability. 

(2)  The  approach  was  broadened  to  include  additional  independent  variables,  measuring 
religious  and  ethnic  diversity,  quality  of  public  health  systems,  international  trade  relationships, 
and  conunitments  to  political  rights. 
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(3)  The  new  FASE  methodology  demonstrated  the  capability  to  validate  a  global,  15 -year 
forecast  of  country  instability  at  about  75  percent  overall  accuracy. 

THE  PROJECT  EFFORT  was  conducted  by  Dr.  Sean  P.  O’Brien,  Resource  Analysis  Division, 
Center  for  Army  Analysis  (CAA). 

COMMENTS  AND  QUESTIONS  may  be  sent  to  the  Director,  Center  for  Army  Analysis, 
ATTN:  CSCA-RA,  6001  Goethals  Road,  Suite  102,  Fort  Belvoir,  VA  22060-5230. 
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1  ANALYSIS  OF  COMPLEX  THREATS  II 

1.1  Introduction 


This  project,  Analysis  of  Complex  Threats  II  (ACT  II),  was  co-sponsored  by  the  Headquarters, 
Department  of  the  Army  (HQDA)  Deputy  Chief  of  Staff  for  Operations  and  Plans  (DCSOPS) 
and  the  Deputy  Chief  of  Staff  for  Intelligence  (DCSINT). 

fl .2  .■■Purpose- '  , 


Purpose 

Develop  and  validate  an  approach  or  methodology 
to  forecast  country  and  regional  instabilities  over 
the  period  2010-15. 

*  Broaden  the  scope  of  the  country  instability 
analysis  reported  \r\  Analysis  of  Complex  Threats 
(ACT)  (CAA-SR-99-4)  to  include  additional 
independent  and  dependent  variables. 

*  Continue  exploration,  application,  development, 
and  refinement  of  analytical  tools  to  explore  the 
relationship  between  country  macro-structurals  and 
country  instability. 


Figure  1.  Purpose 


In  November  1997,  the  Office  of  the  Deputy  Chief  of  Staff  for  Operations  and  Plans 
(ODCSOPS),  War  Plans  Division,  asked  the  Center  for  Army  Analysis  (CAA)  to  develop  and 
demonstrate  a  methodology  to  forecast  the  likelihood  that  some  nonspecific  level  of  instability 
could  occur  in  any  given  country  over  a  5-year  period  that  might  challenge  US  national  security 
interests  and  precipitate  smaller-scale  contingency  (SSC)  deplojmients  by  the  Army.  The  War 
Plans  Division  wanted  an  analytically  defensible  approach  for  supporting  the  development  and 


ACTU 


ANALYSIS  OF  COMPLEX  THREATS  II  •  1 


CAA-R-01-41 


evaluation  of  long-range  scenarios  in  which  the  Army  may  be  deployed  to  defend  and  support 
US  national  security  interests  in  the  future.  This  initial  work  was  completed  by  CAA  in  May 
2000  and  is  described  in  the  report.  Analysis  of  Complex  Threats  (ACT). 

The  ACT  Study  first  drew  upon  several  prior  CAA  studies  to  identify  key  structural  factors  that 
could  contribute  to  the  stability  (or  instability)  of  a  country.  A  country’s  gross  domestic  product 
(GDP)  per  capita,  infant  mortality  rate,  position  on  a  political  rights  index,  youth  bulge,  and  daily 
caloric  consumption  per  person  per  day  were  the  internal  country  factors  used  in  ACT  analytical 
models.  The  relationship  between  these  factors  and  historical  instances  of  country  instability  was 
explored  and  modeled  using  traditional  machine  learning  and  data  mining  techniques  in  addition 
to  more  recently  developed  statistical,  fuzzy,  and  temporal  data  mining  techniques.  For  the 
purpose  of  validating  the  models,  country  instability  was  reflected  by  “armed  conflicts  involving 
at  least  25  battle-related  deaths”  that  occurred  between  1989  and  1997.  The  forecasting 
capabilities  of  ACT  were  validated  and  tested  for  global  and  regional  models  with  good 
performance  results  over  a  5-year  period  of  time. 

At  the  request  of  the  Office  of  the  Deputy  Chief  of  Staff  for  Intelligence  (ODCSINT),  the  ACT 
study  was  briefed  to  LTG  Kennedy  on  13  May  1999.  LTG  Kennedy  asked  CAA  to  further 
develop  and  improve  ACT.  Specifically,  LTG  Kennedy  requested  that  CAA  consider  adding 
factors  to  the  model,  such  as  ethnic/religious  diversity  and  environmental  factors,  continue  to 
develop  new  analytical  tools  and  techniques,  and  broaden  the  definition  of  instability  to  include 
both  violent  and  nonviolent  (but  still  serious)  events  that  characterize  country  instability.  The 
Analysis  of  Complex  Threats  II  (ACT  II)  study,  presented  in  this  report,  evolved  fi’om  this 
tasking.  On  27  May  1999,  MG  St.  Onge  agreed  to  cosponsor  ACT  II  with  the  ODSCINT. 


il.3  ACT  II  Enhancements 


Figure  2  describes  the  enhancements  that  were  made  to  the  country  instability  forecasting 
approach  in  ACT  II  in  comparison  to  the  level  of  sophistication  achieved  in  ACT.  These  include 
the  following; 


•  ACT  possessed  the  capability  to  forecast  the  likelihood  than  an  instability  would  occur  in  any 
given  country  over  a  2-  through  5-  year  period  with  about  70  percent  accuracy.  Currently,  ACT 
n  provides  the  capability  to  forecast  not  just  the  likelihood  that  an  instability  will  occur,  but  also 
that  that  instability  will  occur  within  a  certain  range  or  level  of  intensity  (e.g.,  low,  moderate,  or 
high).  Despite  this  greater  specificity  on  the  element  of  prediction,  it  proved  possible  in  ACT  II 
to  validate  a  15 -year  forecast  at  about  75  percent  accuracy. 

•  Several  factors  (e.g.,  contributors  to  instability  or  independent  variables)  were  added  to  the 
ACT  database.  These  included  information  on  a  country’s  commitment  to  civil  liberties,  level  of 
democracy,  commitment  to  rules  of  global  trade,  its  ethnic  and  religious  diversity,  and  its 
peoples’  average  life  expectancy. 

•  The  ACT  database  contained  annualized  data  for  every  major  country  over  the  9- year  period 
1989-1997.  In  order  to  accommodate  the  need  to  validate  a  15-year  forecast,  the  temporal  frame 
of  the  database  had  to  be  deepened;  currently,  the  ACT  II  database  covers  the  period  1960-1999. 
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ACT  II  Enhancements 


ACT 

ACT  II 

Element  of  prediction 

Likelihood  of  country  instability 

Likelihood  that  country  will 
experience  a  certain  level  of 
intensity  of  instability 

Vaiidation  forecast  horizon 

2  -  5  years  out 

15  years  out. 

Temporal  domain  of 
historical  database 

1989-  1997 

1960-  1999 

Macro-structural  indicators 

5-7  (depending  on  analytical 
technique  used) 

12 

Proxy  indicator  of  instability 
(dependent  variable) 

Violent,  armed  conflict  that 
results  in  battle  fatalities 

Violent  and  nonviolent  crises  and 
wars  that  may  or  may  not  involve 
battle-related  fatalities 

Figure  2.  ACT  II  Enhancements 


•  Finally,  the  definition  of  instability  and  therefore  the  scope  of  the  study  was  expanded  to 
include  both  violent  and  nonviolent  but  still  serious  types  of  wars  and  state  crises.  It  is  prudent 
to  consider  within  an  early  warning  fi"amework,  both  violent  and  nonviolent  conflicts  because, 
short  of  complete  conflict  prevention,  it  is  the  prevention  of  that  escalation  from  nonviolent 
crisis  to  war  that  one  hopes  to  accomplish  with  some  advanced  warning.  The  inclusion  of 
violent  and  nonviolent  events  of  varying  degrees  of  intensity  also  facilitates  the  creation  of  a 
proxy  index  of  instability  which  is  used  in  this  study  as  the  basis  from  which  to  generate 
estimates  of  the  likelihood  of  different  levels  of  intensity  of  country  instability. 
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;Origmal  A  Data 


Figures  3  and  4  describe  the  12  macro-structural  factors—potential  contributors  to  instability- 
used  in  ACT  II  to  validate  a  15-year  forecast.  The  six  macro-structural  indicators  in  Figure  3 
were  used  in  the  ACT  study;  the  six  factors  in  Figure  4  have  since  been  added  to  the  database  for 
use  in  ACT  II  analyses.  Data  were  collected  for  each  of  these  12  factors  on  an  annual  basis  for 
every  major  country  over  the  period  1960-1999.  Generally,  however,  only  data  for  the  period 
1975-1999  are  used  to  validate  the  15-year  forecast.  This  temporal  restriction  was  imposed  for 
two  reasons.  First,  in  general,  the  quality  and  consistency  of  the  data  degrades  the  further  one 
goes  back  in  time.  Second,  recognizing  that  the  nature  and  causes  of  intra-  and  interstate 
conflict  have  probably  evolved  over  time,  it  seemed  prudent  to  ensure  that  an  analysis  of  conflict 
in  and  around  the  post-Cold  War  era  (and  well  beyond  to  2015)  was  not  unduly  influenced  by  the 
nature  and  amount  of  conflict  that  occurred  in  the  distant  past. 


1.  Percent  of  history  spent  in  state  of  conflict;  percent  of  time  (in  years)  spent  in  a  state  of  conflict 
as  defined  by  KOSIMO  (to  include  crises,  violent  crises,  and  wars).  Note:  percent  of  time  in  conflict 
spans  the  years  in  training  data  ONLY.  Source:  KOSIMO  data  project,  Pfetsch  and  Rohloff  (2000). 

2.  Infant  Mortality  Rate:  (1975-1997):  Number  of  deaths  of  children  under  1  year  of  age  per  1,000 
live  births.  Source:  U.S.  Bureau  of  the  Census,  International  Database  (www.census/gov/ipc/www), 

3.  Youth  Bulge  (1975-1997):  Ratio  of  population  aged  15-29  to  those  aged  30-54.  Source:  US  Bureau 
of  the  Census,  International  Data  Base  (www.census/gov/ipc/www). 

4  .  Caloric  Intake  (1975-1997):  Estimate  of  the  average  number  of  calories  consumed  per  person,  per 
day.  Source:  Food  and  Agriculture  Organization  of  the  United  Nations  (http://apps.fao.org). 

5.  GDP  per  capita  (1975-1998):  Annual  gross  domestic  product  per  person  measured  in  constant  1998 
US  dollars.  Source:  World  Bank  (1999).  World  Development  Indicators 

6.  Political  Rights  Index  (1975-1998):  Measure  of  rights  to  participate  meaningfully  in  the  political 
process  (same  scaling  as  for  Civil  Liberties  Index).  Source:  Freedom  House  (www.freedomhouse.org). 


Figure  3.  Original  ACT  Study  Data 
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7.  Life  Expectancy  f 1975"  1997V  Average  life  expectancy  (males  and  females  combined) 
Source:  US  Bureau  of  the  Census,  International  Data  5a5e(www.census/gov/ipc/www). 

8.  Civil  Liberties  index  (1975-1998):  Measure  of  the  freedom  of  country’s  people  “to  develop 
views,  institutions,  and  personal  autonomy  apart  from  the  state ’’  Seven  point  ordinal  scale  with 
l=free,  7=not  free.  Source:  Freedom  House  fwwav.freedomhouse.orgl 

9.  Trade  Openness  (1975-1998):  Value  of  a  country’s  total  imports  and  exports  as  a  percent  of 
GDP  per  capita.  Source-  PENN  World  Tables  {\915-\992)\  1999  World  Bank  Development 
Indicators  (1993-1998). 

10-  Democracy  fl975- 1998):  Measure  of  degree  of  democracy;  ranges  from-  01  (least 
democratic)  to  10  (most  democratic).  Sources:  Polity98  project  (Gurr  and  Jaggers  1995; 
Gleditsch  and  Ward  1997;  http://k:leditsch.socsci.gla.ac.uk/Polity.html);  (see  also  Marshall  and 
Jaggers  n.d.  for  a  recent  update). 

IL  Religious  Diversity  (1975-1999):  Largest  religious  group  in  country  as  a  percent  of  total 
population.  Sources:  CIA  World  Fact  Book;  Country  Indicators  of  Foreign  Policy  Projed 
(CIFP)\  Ellingsen  (1996);  Handbook  of  the  Nations',  Britannica  Book  of  the  Year;  Demographic 
Yearbook. 

12>  Ethnic  Diversity  (1975-1999):  Largest  ethnic  group  in  country  as  a  percent  of  total 
population.  Sources:  same  sources  used  to  measure rgfagfoiis  diversity  above. 


Figure  4.  Data  Added  for  ACT  II 
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1.6  Expand^ing  the  beflnitioii  of  Instabilit  y 


Figure  5  describes  how  the  definition  of  instability  was  broadened  in  ACT  II  vis-a-vis  ACT.  The 
ACT  study  used  armed  conflicts  that  involved  battle  fatalities  as  the  proxy  measure  of  instability 
used  to  validate  the  macro-structural  factors  as  relevant  contributors  to  instability.  These  data 
were  acquired  from  the  Conflict  Data  Project  at  Uppsala  University.  The  Uppsala  Data  Project 
tracks  all  violent  conflicts  that  have  transpired  aroimd  the  world,  and  in  which  at  least  25  battle 
fatalities  occurred,  over  the  period  1989-1997  (it  has  since  been  updated  through  1999).  Though 
it  proved  to  be  adequate  given  the  purpose  of  the  ACT  study,  this  data  set  was  not  sufficient  to 
meet  the  requirements  for  ACT  II.  First,  its  temporal  limitation  (1989-1999)  prohibits  its  use  in 
validating  a  15-year  forecast.  Second,  it  contains  information  only  on  violent  intra-  and  inter¬ 
state  conflicts  that  are  rare  events  relative  to  the  number  of  less  violent  yet  still  serious  inter-  and 
intrastate  interactions.  If  left  to  fester  in  the  absence  of  remedial  intervention,  it  is  these  less 
violent,  but  conflictual  interactions  that  have  the  potential  to  escalate  into  far  more  serious  state 
crises  and  wars.  As  such,  the  inclusion  of  these  nonviolent  crises  in  a  framework  that  seeks  to 
anticipate  serious  state  crises  and  instabilities  is  a  desirable,  if  not  crucial,  element  in  its  potential 
to  achieve  a  measure  of  success. 


Expanding  the  Definition  of  “instabiiity" 


ACT  dependent  variable 


The  likelihood  that  any  given  country  would  experience  a  conflict 
annually  over  a  2-  through  5-year  period. 


Conflict  defined  “An  armed  conflict  is  a  contested  incompatibility  which 

concerns  government  and/or  territory  where  the  use  of  armed 
force  between  two  parties,  of  which  at  least  one  is  the 
government  of  a  state,  results  in  at  least  25  battle-related 
deaths.” 

{Source:  Uppsala  Conflict,  Conflict  Data  Project:  States 
in  Armed  Conflict,  Uppsala  University,  Sweden) 


The  likelihood  that  any  given  country  would  experience  a  certain 
ACT  II  dependent  variable  — >  Qf  intensity  of  instability  annually  over  a  1 5-year  period 

conflict  over  a  2-year  period  through  5-year  period. 


Conflict  defined:  “....[the]  clashing  of  overlapping  interests  (positional 
differences)  around  national  values  and  issues  (independence,  self  - 
determination,  borders,  and  territory,  access  to  or  distribution  of 
domestic  or  international  power);  the  conflict  has  to  be  of  some  duration 
and  magnitude  of  at  least  two  parties  (states,  groups  of  states, 
organizations,  or  organized  groups)  that  are  determined  to  pursue  their 
interests  and  win  their  case.  At  least  one  party  is  the  organized  state. . .  ” 


{Source:  Heidelberg  Institute  of  International  Conflict 
Research  (HIIK),  Heidelberg,  Germany) 


Figure  5.  Expanding  the  Definition  of  Instability 
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It  is  for  these  reasons  that  an  alternative  data  set  was  used  in  ACT  II.  The  KOSIMO  project  at 
the  Heidelberg  Institute  of  International  Conflict  Research  in  Germany  embraces  a  deliberately 
vague  definition  of  con/7ict~essentially,  any  incompatibility  between  at  least  one  state  and 
another,  or  between  at  least  one  state  and  some  non-state  group  that  lasts  for  a  while  and  is 
somewhat  intense  (see  Figure  5  for  the  project’s  formal  definition). 

The  KOSIMO  project  constructed  a  comprehensive  database  of  all  conflicts  occurring  between 
1945  and  1999,  regardless  of  type  (e.g.,  interstate  wars,  intrastate  wars,  foreign  policy  crises, 
etc.)  and  intensity.  Many  of  the  violent  conflicts  were  drawn  from  well-known  data  collections 
such  as  the  Correlates  of  War  (COW)  project  at  the  University  of  Michigan  (recently  moved  to 
Penn  State  University)  and  the  International  Crisis  Behavior  (ICB)  project  at  the  University  of 
Maryland  among  others.  However,  KOSIMO  also  allows  for  nonviolent  conflicts  specifically 
because  they  have  the  potential  to  escalate  into  violent  conflicts. 

The  KOSIMO  project  identified  74  mostly  nonviolent  crises,  121  violent  crises,  and  61  wars 
over  the  period  1975-1999.  Figure  5  defines  and  provides  examples  of  each  of  the  three  conflict 
types.  Using  the  data  from  KOSIMO,  a  score  (ranging  from  1  to  4)  was  assigned  for  each 
country- year  in  the  database  according  to  the  maximum  level  or  intensity  of  conflict  the  country 
experienced  in  that  year.  If  the  most  intense  conflict  in  which  a  country  engaged  was  a  war, 
either  as  an  initiator  or  as  a  defender,  then  it  receives  a  score  of  4  for  that  year.  It  receives  a 
score  of  3  if  it  experienced,  at  most,  a  violent  crisis,  a  2  if  it  experienced  no  more  than  a 
nonviolent  crisis,  and  a  1  if  it  experienced  none  of  these  three  conflict  event  types.  This  four- 
category  ordinal  level  scale  is  the  proxy  measure  of  instability  used  to  validate  the  factors 
described  above  as  relevant  contributors  to  instability. 

Based  on  the  historical  relationship  between  country  macro-structurals  and  the  proxy  index  of 
instability  as  uncovered  in  the  training  data,  the  Fuzzy  Analysis  of  Statistical  Evidence  (FASE) 
algorithm  used  to  analyze  these  factors  computes  the  likelihood  that  each  of  these  four  conflict 
events  will  occur  annually  in  each  country  in  the  test  set.  These  likelihood  measures  are 
aggregated,  and  an  expected  intensity  level  of  instability  is  derived  for  each  country  by  applying 
the  following  decision  rules: 

1.  If  the  combined  probabilities  of  conflict  types  1  and  2  occurring  in  a  given  country  is 
greater  than  67  percent,  then  the  expectation  is  that  the  country  will  experience  no  or  low 
intensity  instability. 

2.  If  the  combined  probabilities  of  conflict  types  2  and  3  occurring  in  a  given  country  is 
greater  the  67  percent,  then  the  exception  is  that  the  country  will  experience  a  moderate  intensity 
instability. 

3.  If  the  combined  probabilities  of  conflict  types  3  and  4  occurring  in  a  given  country  are 
greater  the  67  percent,  then  the  exception  is  that  the  country  will  experience  a  high  intensity 
instability. 

4.  If  more  than  one  of  the  first  three  decision  rules  allies  to  a  particular  country-year 
forecast,  select  the  one  that  reflects  the  highest  intensity  level  of  instability. 
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1.7  Three  Levels  of  Instability  Intensity 


Instabilitv  Levels 

Confiict  Tvoe 

Examoles 

Definition 

High  intensity  ^ 

^  4-War 

WWII,  Guif  War,  Six  Days  War 

Systematic,  collective  use  of 
force  by  regular  troops 

1 

Moderate  intensity 

^  3- Violent  crisis 

r 

Northern  Ireland,  Basque 
separatists,  ethnic  conflict  in 
Bosnia 

Sporadic,  irregular  use  of 
force,  “war-in-sight”  crises 

None/Low  intensity  -< 

^  2-  Crisis 

Russian  Federation  vs 

Ukraine  over  possession  of 
strategic  weapons 

Mostly  nonviolent 

1-  None 

Figure  6.  Three  Levels  of  Instability  Intensity 


5.  If  none  of  these  decision  rules  applies,  that  is,  if  the  probability  of  any  one  of  the  three 
instability  intensity  levels  is  roughly  equally  distributed  across  the  possible  conflict  event  t3q)es, 
then  we  are  uncertain  about  that  country’s  likelihood  of  instability.  (An  uncertain  forecast  is 
neither  correct  nor  incorrect  as  far  as  the  performance  of  the  forecasting  algorithm  is  concerned. 
In  any  given  year,  approximately  10-15  percent  of  all  predictions  made  are  of  an  uncertain 
nature.  Historically,  about  50  percent  of  these  cases  ultimately  experience  a  conflict  type  2,  3,  or 
4  in  that  year.) 

6.  A  forecast  is  correct  if,  and  only  if,  a  cormtry  experiences,  as  its  maximum  intensity  level 
of  conflict,  one  of  the  two  conflict  type  events  covered  by  the  forecasted  level  of  instability 
intensity,  as  defined  in  Decision  Rules  1-3. 

In  addition  to  the  ease  with  which  they  facilitate  the  presentation  and  interpretation  of  results, 
these  decision  rules  were  developed  because  they  also  facilitate  an  arguably  fair,  and  somewhat 
conservative,  test  of  the  forecasting  method’s  performance.  The  sixth  decision  rule,  for  instance, 
articulates  the  conditions  under  which  a  forecast  will  be  considered  correct.  If  the  algorithm 
correctly  forecasts  that  a  country  will  be  unstable,  but  does  not  identify  the  correct  intensity  level 
of  that  instability,  then  the  forecast  is  considered  incorrect.  For  example,  if  the  algorithm  were  to 
forecast  that  Country  A  in  1991  had  an  86  percent  probability  of  experiencing  a  moderate 
intensity  instability  (e.g.,  either  conflict  type  2  or  3),  and  the  country  actually  experienced  a  war 
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that  year  (conflict  type  4)  then  that  forecast  would  be  considered  a  miss  and  therefore,  incorrect. 
Of  course,  other  decision  rules  may  be  equally  plausible. 

In  the  ACT  study  (CAA-SR-99-4),  analysts  evaluated  several  different  analjdical  techniques, 
including  logistic  regression,  classification  and  regression  trees  (CART),  temporal  decision  trees, 
and  neural  networks  to  determine  how  well  each  could  identify  or  “learn”  patterns  in  the 
relationships  between  coimtry  macro-structurals  and  the  likelihood  of  country  instability.  The 
technique  that  consistently  demonstrated  an  ability  to  accurately  distinguish  those  countries  that 
would  and  those  that  would  not  subsequently  experience  an  instability,  given  the  values  of 
country  macro-structurals,  was  one  that  was  developed  by  the  Center  for  Army  Analysis 
specifically  for  these  types  of  classification  problems.  FASE  is  a  nonlinear,  nonparametric 
pattern  classification  algorithm.  It  is  a  hybrid  technique  that  incorporates  theoretical  elements 
from  statistics,  fuzzy  logic,  and  possibility  theory.  FASE  is  used  in  the  ACT  II  study  to  validate 
a  15-year  forecast.  A  detailed  description  of  FASE  can  be  found  in  the  ACT  report. 


1.8  Forecasting  Method 


Fuzzy  Analysis  of  Statistical  Evidence  (FASE) 
Model 

-  Nonlinear,  nonparametric,  pattern 
classification  algorithm. 

-  Hybrid  method,  incorporating  elements  from 
statistics,  fuzzy  logic,  and  possibility  theory. 

-  Developed  by  CAA  specifically  for  ACT- 

related  applications. 

-  Professionally  peer-reviewed;  patent  pending 

Chen,  YuanYaa  2000.  “Fuzzy  Analysis  of  Statistical  Evidence.” 
IEEE  Transactions  on  Fuzzy  Systems  8(6):796-799 


Figure  7.  Forecasting  Method 


Several  enhancements  were  made  to  FASE  between  the  ACT  and  ACT  II  studies.  For  example, 
the  early  version  of  FASE  used  in  ACT  could  accommodate  only  limited  dependent  variables 
with  two  classes  (e.g.,  stable/unstable,  conflict/no  conflict).  The  revised  methodology  can 
predict  to  multiple  classes  on  the  dependent  variable.  Second,  the  early  version  of  FASE 
developed  predictions  of  coimtry  instability  using  a  combination  of  the  possibility  measure  and 
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certainty  factor  which,  though  mathematically  rigorous,  are  new  and  imconventional  likelihood 
measures.  The  new  FASE,  while  still  based  on  the  possibility  scores  and  certainty  factors, 
facilitates  the  transformation  of  this  inference  of  classification  into  probabilities  that  are  more 
familiar  and  more  easily  interpreted.  Finally,  the  new  FASE  incorporates  control  variables  to 
facilitate  the  development  of  more  nuanced  models  of  country  instability.  If  the  statistical 
patterns  of  conflict  vary  significantly  by  a  particular  independent  variable--for  instance,  by 
geographic  region  or  level  of  economic  development— then  this  independent  variable  can  be  used 
as  a  control  variable  in  the  analysis. 


1.9  Enhancements  Made  to  FASE  Model  for  ACT  11 


•  The  dependent  variable  extended  from  binary  ciasses  to 
accommodate  muitiple  ciasses. 

‘  Transformed  the  inference  of  classification  into 
probability  measures  for  forecast  prediction. 

•  Added  control  variable  to  incorporate  different  statistical 
patterns. 


Figure  8.  Enhancements  Made  to  FASE  Model  for  ACT  II 

It  may  be  instructive  to  illustrate  the  application  of  FASE  by  example.  Consider  the  following 
illustration:  Let  D  be  some  dependent  variable  with  four  classes  {Dl,  D2,  D3,  Df).  A  set  of 
attributes,  Al,  A2,  A3,  which  can  take  a  range  of  values,  are  thought  to  be  correlated  with  D  and 
available  in  the  form  of  a  historical  database.  Given  a  set  of  observations  on  Aj,  A2,  A3,  for 
some  transaction  (or  case)  T,  evaluated  in  a  historical  context,  what  is  the  likelihood  of  each  of 
the  four  classes  of  D?  This  question  is  analogous  to  predicting  the  correct  instability  intensity 
level  from  a  set  of  country  macro-structural  attributes.  Using  FASE,  we  approach  the  problem  as 
follows.  First,  we  divide  the  data  set  into  training  and  test  sets  using  spatial  or  temporal  rules 
and  then  either  randomly  or  by  selection.  Second,  we  split  the  data  for  each  attribute  in  the 
training  set  by  each  class  label  of  D  and  estimate  the  class  probability  distributions  for  each 
attribute.  For  discrete  variables,  the  probability  is  estimated  by  the  relative  frequency  in  each 
category.  For  continuous  variables,  we  estimate  the  probability  density  function  using  an 
average  shifted  histogram  (ASH),  kernel  method,  or  other  suitable  density  estimator.  These 
probability  distributions  are  the  likelihood  templates  against  which  observations  on  the  attributes 
in  the  test  set  are  to  be  evaluated. 
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1.10  FASE:  Possibility  Aggregation  Example 


FASE:  Possibility  Aggregation  Example 


Di 

D2 

Da 

D4 

50%  (1) 

10%  (.2) 

5%  (.1) 

15%  (.3) 

25%(.71) 

10%  (.29) 

5%  (.14) 

35%  (1) 

missing  (1) 

missing  (1) 

missing  (1) 

missing  (1) 

0.71 

0.13 

0.04 

0.30 

1 

0.18 

0.06 

0.42 

60% 

11% 

4% 

25% 

Note:  Probabilities  [P(Ai|Dj)]are  in  the  cells;  possibility  scores  are  in  parentheses.  D,  is  most 
likely  of  the  four  :lass  labels. 


T  (a,  b. . .  .0  =  log  f  l+Cs^-OCs”-!) . (s‘-l)/(s- 


Where; 


•k  =  number  of  attributes  (e.g.  independent  variables) 
•0<5<1 


Figure  9.  FASE:  Possibility  Aggregation  Example 


Third,  using  data  in  the  test  set,  we  evaluate  how  well  the  algorithm  can  classify  on  each  of  the 
class  labels  on  D.  To  do  that,  for  any  observation  on  an  attribute  in  the  test  set,  we  calculate  the 
likelihood  ratio  across  the  class  labels  on  D,  based  on  the  patterns  observed  in  the  class 
probability  distributions  from  the  training  set.  Suppose  those  probabilities  are  distributed  across 
the  class  labels  as  they  appear  in  the  example  in  Figure  9.  Historically,  in  this  example, 
transactions  with  values  on  the  order  of  magnitude  comparable  to  those  for  A]  and  A2  have  been 
associated  with  the  outcome  Dl  50  percent  and  25  percent  of  the  time,  respectively  (note  the  data 
on  A3  are  completely  missing  for  purposes  of  illustration). 


If  and  A2  are  not  independent  (and  this  is  the  assumption  here),  then  we  cannot  aggregate  the 
evidence  across  them  to  compute  the  likelihood  of  the  class  labels  without  violating  the 
assumptions  of  probability  theory.  Therefore,  in  the  fourth  step,  we  normalize  the  likelihood 
ratios  for  each  attribute  into  a  possibility  measure,  using  the  most  likely  class  label  as  the  base. 
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The  possibility  measxires  appear  in  each  cell  in  parentheses.  Thus,  by  definition,  the  most  likely 
class  label  for  any  observed  attribute  value  will  always  have  a  possibility  score  of  1.  A 
possibility  score  of  1  is  also  given  in  those  instances  in  which  the  data  on  an  attribute  for  some 
transaction  is  missing.  Since,  in  such  instances,  no  class  label  is  any  more  or  less  likely  than  any 
other,  this  ensures  that  missing  data  are  treated  as  such.  The  possibility  measure  is  not  neatly 
interpretable;  one  must  consider  the  possibility  and  its  conjugate  (the  belief  measure)  together.  A 
possibility  of  1  does  not  represent  complete  certainty  about  an  outcome  as  it  does  in  probability, 
but  is  rather  only  an  imprecise  indication  of  our  belief  in  that  outcome,  relative  to  the  other 
possibilities;  however,  the  possibilities  do  reflect  ordinal  properties  consistent  with  probability 
theory. 

In  the  fifth  step,  we  aggregate  the  possibility  measures  across  the  attributes  for  each  class  label 
on  D  using  the  fiizzy  set  t-norm  known  as  the  Frank  Rule. 

T  (a,  h....i)  =  logs(l+(sa-l)(sb-l) . (si- l)/(s- 1)^-1); 

where  k  is  the  number  of  attributes  and  s  is  an  adjustment  parameter  that  is  set  close  to  0  if  our 
independent  variables  are  highly  correlated  and  1  if  they  are  independent  (5  is  set  to  .01  here). 

We  apply  the  Frank  Rule  to  the  possibility  scores  for  the  attributes  on  each  class  label.  This 
produces  overall  likelihood  ratios,  which,  again,  are  normalized  into  overall  possibility  measures 
for  each  class  label.  The  overall  possibility  measwe  of  a  class  label  indicates  that  class  label’s 
likelihood  given  a  vector  of  observed  (or  forecasted)  attribute  values.  For  ease  of  interpretation, 
these  overall  possibility  scores  for  each  class  on  the  dependent  variable  can  be  transformed  back 
into  probability  measures  by  straightforward  normalization.  Having  applied  this  process  to  the 
example  in  Figure  9,  we  see  that  Df  with  a  60  percent  probability,  is  the  most  likely  outcome. 
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‘i.il  Model  Development,  Training,  and  Validation 


r:  Model  DevelopmentITraining  data:  1975-1984 

•  Algorithm  “learns”  how  structural  independent 
variables  have  been  associated  with  different  levels 
of  conflict  historically. 

■  Model  ValidatlonITest  data:  1985-1999 

•  Program  algorithm  to  predict  probability  and  intensity 
of  conflict  based  on  values  of  observed  independent 
variables  and  patterns  “learned”  in  training  set. 

•  Compare  algorithm’s  predictions  to  historical  record; 
compute  performance  metrics  (accuracy,  recall, 
precision). 


Figure  10.  Model  Development,  Training,  and  Validation 


FASE  was  used  in  ACT  II  to  validate  a  15-year  global  forecast  of  country  instability.  To  do  so,  a 
split-sample  validation  design  was  used.  The  data  for  the  10-year  period  1975-1984  are  used  as 
the  training  set.  The  FASE  procedure  was  applied  to  the  data  in  this  training  period  to  “learn” 
how  different  configurations  and  levels  of  country  macro-structurals  have  been  associated  with 
different  levels  of  instability.  Then,  using  the  data  on  country  macro-structurals  only  for  the 
period  1985-1999  (the  test  set),  FASE  classified  the  countries  by  their  expected  intensity  levels 
of  instability  based  on  the  historical  patterns.  We  then  compare  how  FASE  classified  each 
country  with  actual  occurrences  over  the  period  1985-1999  (as  defined  by  KOSIMO)  and 
compute  standard  performance  metrics.  The  performance  metrics  reveal  how  well  FASE  can 
learn  the  existing  patterns  and  how  robust  the  patterns  are  through  time. 
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1.12  Model  Development,  training,  and  Validation  -  Colombia 


Possibility  scores  and  forecast  for  Colombia  (1992)  using  patterns  observed  in  data  from  1975-1984. 


Variable 

Observed  value 

PosMolent  crisis) 

Caloric  intake 

2443 

1 

0.74 

0.53 

0.81 

Infant  mortality  rate 

33.00 

0.71 

1 

0.51 

0.19 

Political  rights  index 

2 

0.73 

1 

0.52 

0.4 

GDP  per  capita 

2007.29 

0.83 

0.8 

0.86 

1 

Civil  Liberties  index 

4 

0.99 

0.94 

1 

0.53 

68 

0.62 

1 

0.55 

0.22 

Youth  Bulae 

1.12 

0.75 

1 

0.41 

0.86 

Reliaious  Diversity 

95 

0.82 

1 

0.92 

0.7 

Ethnic  Diversitv 

58 

1 

0.97 

0.56 

0.43 

Democracy 

9 

1 

0.8 

0.82 

0.16 

Trade  Openness 

35 

0.67 

1 

0.89 

1 

BISH 

70% 

0.14 

0.78 

0.7 

1 

0.11 

0.54 

0.17 

0.03 

Overall  possibilitv 

0.20 

1 

0.31 

0.06 

Overall  probability 

13% 

64% 

20% 

4% 

Predicted  level  of  instability: 

84%  likelihood  of  conflict  type  2  or  3  (moderate  intensity) 

Conflict  type  that  occurred: 

Violent  crisis  (Type  3) 

Accuracy  of  forecast 

Correct  prediction 

Figure  11.  Model  Development,  Training,  and  Validation  -  Colombia 


Figures  1 1  and  12  display  output  of  this  validation  analysis  for  Colombia  in  1992  and 
Kyrgyzstan  in  1994.  These  forecasts  were  generated  from  patterns  FASE  identified  over  the 
period  1975-1984  and  therefore  represent  8  and  10-year  validation  forecasts,  respectively. 

Figures  10  and  1 1  are  designed  to  resemble  the  notional  example  in  Figure  8.  The  second 
column  in  each  table  displays  the  observed  values  for  each  of  the  macro-structural  factors  for 
that  country  in  the  year  indicated.  The  possibility  scores  associated  with  each  of  the  four  class 
(or  conflict  type)  outcomes  are  displayed  in  the  cells. 

The  results  in  Figure  1 1  suggest  that,  based  on  the  macro-structural  factors  Colombia  exhibited 
in  1992  and  the  decision  rules  articulated  above,  we  would  expect  a  moderate  intensity  instability 
(conflict  type  2  or  3)  to  occur  with  an  84  percent  probability.  Colombia  was  engaged  in  a  violent 
crisis  (conflict  type  3)  in  1992— a  counterinsurgency  against  the  Revolutionary  Armed  Forces  of 
Colombia  (FARCII)  that  began  in  1964  and  persists  today— and  nothing  more  serious,  so  we  would 
regard  this  forecast  as  a  correct  prediction. 
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j|.13  Model  Development,  Validation  -  Kyrgyzirtan 


Possibility  scores  and  forecast  for  Kyrgyzstan  (1994)  using  patterns  observed  in  data  from  1975-1984. 


Observed  value 

mm/smsu 

Caloric  intake 

2322 

0.84 

0.88 

0.82 

1 

Infant  mortality  rate 

67 

0.48 

0.26 

1 

0.84 

4 

0.52 

0.37 

1 

0.35 

GDP  per  capita 

840.72 

0.79 

0.76 

0.86 

1 

Civil  Liberties  index 
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Figure  12.  Model  Development,  Training,  and  Validation  -  Kyrgyzstan 


Kyrgyzstan  is  an  interesting  ease  because  it  does  not  even  exist  in  the  period  covered  by  the 
training  set.  Also,  data  were  available  on  Kyrgyzstan  only  for  half  the  test  set  years  (1991- 
1999).  Therefore,  its  percent  of  history  in  state  of  conflict  variable,  which  is  calculated  based 
only  on  years  in  the  training  set,  is  completely  missing.  KOSIMO  does  not  record  a  conflict  for 
Kyrgyzstan  in  1994  or,  for  that  matter,  for  any  other  year  between  1991  and  1999.  The  FASE 
analysis  does  indicate  that  based  on  Kyrgyzstan’s  macro- structurals  in  1994,  a  conflict  type  1 
(none)  is  most  likely  (36  percent  probability).  However,  the  probabilities  on  no  two  adjacent 
conflict  levels  breach  the  67  percent  threshold.  So,  by  the  strict  decision  rules  governing  this 
analysis,  we  could  conclude  that  we  are  uneertain  about  what  level  of  instability  Kyrgyzstan  was 
likely  to  experience  in  1994,  a  forecast  that  would  be  considered  neither  correct  nor  incorrect 
from  an  overall  performance  perspective. 
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1.14  5- to  15-Year  VaUdation  of  EASE  Model 

Figure  13  displays  the  average  5-  to  15-year  validation  performance  results  calculated  in  5-year 
increments  over  all  159  countries.  Three  performance  metrics  are  computed,  and  their  formulas 
are  displayed  in  Figure  12.  Overall  accuracy  measures  the  ability  of  the  algorithm  to  correctly 
distinguish  between  those  countries  that  do  and  those  that  do  not  experience  some  specified 
intensity  level  of  instability.  The  recall  score  pertains  to  the  ability  of  the  algorithm  to  correctly 
forecast  or  classify  on  the  element  of  interest— in  this  case,  the  correct  conflict  type.  Precision 
refers  to  the  ability  of  the  algorithm  to  classify  without  producing  a  large  number  of  false 
positives  (e.g.,  conflicts  or  instabilities  that  are  forecast  to  occur  but  do  not).  Taken  together,  the 
recall  and  precision  scores  are  generally  the  most  important  and,  ideally,  both  should  be  as  large 
as  possible. 


Global  Validation  (159  countries) 
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Set 
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#of 

years  in 
test  set 
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#  of  predictions  made 
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#  of  conflicts  that 
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Precision 
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predicted  conflicts 
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predicted  to  occur 

1995-99 
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90% 

88% 

82% 

1990-99 

10 

84% 

77% 

77% 

1985-99 

15 

82% 

77% 

70%  s 

Figure  13.  5-  to  15-Year  Validation  of  FASE  Model 


Intuitively,  we  might  expect  the  performance  of  a  method  for  forecasting  conflict  and  instability 
to  degrade  over  time,  which  is  what  we  see  in  Figure  13.  All  the  performance  metrics  trend 
downward  over  the  15-year  forecast  horizon.  Nevertheless,  the  performance  metrics  are 
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promising.  The  FASE  Model  correctly  identified  and  classified  88  percent  of  all  the  conflicts 
that  occurred  in  the  first  5 -year  forecast  horizon  and  did  so  without  producing  many  false 
positives  (82  percent  of  the  cases  predicted  by  the  model  to  be  conflicts  were  not  false  positives). 
The  average  overall  accuracy  and  recall  scores  remain  above  80  percent  and  75  percent, 
respectively,  over  the  entire  15-year  validation  forecast  horizon. 


1.15  15-Year  VaUdation  of  FASE  Model 


Figure  14  displays  the  performance  metrics  broken  out  by  area  of  responsibility  (AOR).  Those 
results  suggest  that  the  FASE  Model  does  very  well  in  predicting  variations  in  levels  of 
instability  for  countries  within  the  United  States  Army  Pacific  Command  (USPACOM),  United 
States  Army  Central  Command  (USCENTCOM),  and  United  States  Army  Southern  Command 
(USSOUTHCOM)  AORs.  Well  over  two-thirds  of  all  the  conflicts  in  which  countries  in  those 
AORs  participated  over  the  period  1985-1999  were  correctly  identified  and  classified  by  the 
level  of  intensity  at  which  they  occurred. 


Average  Validation  Scores  by  Command  (1985-1999) 
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Approach:  Model  Development  and  Validation 

Figure  14.  15-Year  VaUdation  of  FASE  Model 

The  results  are  somewhat  less  impressive  for  countries  in  the  United  States  Army  Atlantic 
Command  (USACOM)  and  United  States  Army  European  Command  (USEUCOM).  USACOM 
includes  only  three  countries— Canada,  Mexico,  and  the  United  States— and  the  model  did  not 
correctly  predict  the  level  of  instability  Mexico  experienced  as  a  result  of  the  Chiapas  Rebellion 
(1994  through  1999).  This  accounts  for  the  preponderance  of  the  poor  performance  witnessed 
within  the  USACOM  AOR.  In  the  case  of  USEUCOM,  the  European  countries  such  as  Britain, 
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France,  and  Balkan  nations  experienced  more  conflict  than  might  be  expected,  given  the 
configuration  of  their  macro-structural  indicators,  whereas  those  African  countries  that  fall 
within  the  AOR  experienced  less  conflict  than  might  be  expected,  given  their  structural 
performance.  The  end  result  is  a  combination  of  relatively  poor  recall  and  poor  precision. 


[Ll6  Summary 


•  ACT  II  provides  a  defensible  analytical  approach  for  developing 
Iona  term,  global,  “first  cut”  forecasts  of  country  instability. 

•  Amenable  to  scenario  development  and  “what  if  drills. 

•  Provides  an  assist  to  strategic  planners. 

•  ACT  II  methodology  is  being  applied  to  help  determine  where  best 
to  place  Army  prepositioning  equipment  sets 


Figure  15.  Summary 


In  summary,  the  ACT  II  study  extended  an  earlier  methodology  to  successfully  validate  a  15- 
year  global  forecast  of  levels  of  coimtry  instability  with  good  performance  results.  The  forecasts 
of  country  instability  are  generated  from  the  patterns  exhibited  by  country  macro-structurals. 
Because  macro-structurals  trend  in  gradual,  largely  predictable  directions,  they  can  be  forecast 
into  the  future  with  a  reasonable  degree  of  expected  accuracy.  This  suggests  the  utility  of  the 
approach  for  conducting  consistent,  rigorous,  “first-cut”  forecasts  of  future  country  instabilities 
in  every  major  country  in  the  world.  These  forecasts  would  be  amenable  to  scenario  development 
and  “what  if’  analyses  since  the  forecasts  of  country  macro-structurals  could  be  readily  adjusted 
to  account  for  new  information  or  changes  in  assumptions. 


It  is  partly  for  these  reasons  that  in  May  2000,  the  Center  for  Army  Analysis  was  asked  by  the 
ODCSOPS  War  Plans  Division  to  apply  the  ACT  II  methodology  in  support  oftwo  logistics 
studies  for  the  Quadrennial  Defense  Review  (QDR):  Enabling  Strategic  Responsiveness  (ESR) 
and  Deployment  Optimization  Research  in  Tools  and  Operations  (DORITO).  The  ESR  and 
DORITO  studies,  taken  together,  will  develop  and  apply  a  methodology  to,  among  other  things, 
determine  where  best  to  preposition  Army  equipment  in  support  of  the  Chief  of  Staff  of  the 
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Army  (CSA’s)  deployment  goals.  Of  considerable  concern  to  ESR  and  DORITO  sponsors  is  the 
question  of  where  challenges  to  US  national  security  interests  might  occur  in  the  future. 
Analyzing  Complex  Threats  for  Operations  and  Readiness  (ACTOR)  will  apply  the  ACT  II 
methodology  to  generate  15-year  (fiscal  year  (FY)  2001-15)  global  forecasts  of  country 
instability.  In  so  doing,  it  will  seek  to  identify  those  countries  that  are  likely  to  possess  in  the 
future  not  only  the  willingness  and  opportunity  to  challenge  US  national  security  interests,  but 
also  the  capacity  to  do  so. 
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APPENDIX  B  REQUEST  FOR  ANALYTICAL  SUPPORT 


P  Performing  Division:  RA  Account  Number:  99167 

A  Tasking:  Verbal  Mode  (Contract-Yes/No):  No 

R  Acronym:  ACT-2 
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Title:  Analysis  of  Complex  Threats  2 

1  Start  Date:  20-Sep-99  Estimated  Completion  Date:  30-Sep-00 

Requestor/Sponsor  (Le.,  DCSOFS):  DCSOPS,  DCSINT  Sponsor  Division:  SSW,  ZA 

Resource  Estimates:  a.  Estimated  PSM:  48  h  Estimated  Funds:  $0.00 
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DCSOPS  and  DCSINT  requested  CAA  to  use  the  Analysis  of  Complex  Threats  methodology  for  use  in  the  Total 
Army  Analysis  -  2009  (TAA-09)  and  the  next  Quadrennial  Defense  Review  (QDR)  to  (1)  conduct  and  validate  a 
10-year  global  forecast  for  the  years  from  2000  through  2010;  (2)  broaden  the  scope  of  the  country  instability 
analysis  to  include  additional  independent  and  dependent  variables;  (3)  explore  the  integration  of  ACT  and  SADE 
analyses;  (4)  continue  the  examination  of  available  methods  and  the  development/refinement  of  new  methods  for 
country  instability  assessment  and  analysis. 

Study  Director/POC  Signature:  Original  Signed  PhoneU: 

Study  Director/POC: 

If  this  Request  is  for  an  Externai  Project  expected  to  consume  6  PSM  or  more,  Part  2  Information  is  Not 
Required.  See  Chap  3  of  the  Project  Directors '  Guide  for  preparation  of  a  Formai  Project  Directive. 
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